Neuralangelo: High-Fidelity Neural Surface Reconstruction

Neural surface reconstruction has been shown to be powerful for recovering dense 3D surfaces via image-based neural rendering. However, current methods struggle to recover detailed structures of real-world scenes. To address the issue, we present Neuralangelo, which combines the representation power of multi-resolution 3D hash grids with neural surface rendering. Two key ingredients enable our approach: (1) numerical gradients for computing higher-order derivatives as a smoothing operation and (2) coarse-to-fine optimization on the hash grids controlling different levels of details. Even without auxiliary inputs such as depth, Neuralangelo can effectively recover dense 3D surface structures from multi-view images with fidelity significantly surpassing previous methods, enabling detailed large-scale scene reconstruction from RGB video captures.Comment: CVPR 2023, project page: https://research.nvidia.com/labs/dir/neuralangel

Fully Immersive Virtual Reality for Skull-base Surgery: Surgical Training and Beyond

Purpose: A virtual reality (VR) system, where surgeons can practice procedures on virtual anatomies, is a scalable and cost-effective alternative to cadaveric training. The fully digitized virtual surgeries can also be used to assess the surgeon's skills using measurements that are otherwise hard to collect in reality. Thus, we present the Fully Immersive Virtual Reality System (FIVRS) for skull-base surgery, which combines surgical simulation software with a high-fidelity hardware setup. Methods: FIVRS allows surgeons to follow normal clinical workflows inside the VR environment. FIVRS uses advanced rendering designs and drilling algorithms for realistic bone ablation. A head-mounted display with ergonomics similar to that of surgical microscopes is used to improve immersiveness. Extensive multi-modal data is recorded for post-analysis, including eye gaze, motion, force, and video of the surgery. A user-friendly interface is also designed to ease the learning curve of using FIVRS. Results: We present results from a user study involving surgeons with various levels of expertise. The preliminary data recorded by FIVRS differentiates between participants with different levels of expertise, promising future research on automatic skill assessment. Furthermore, informal feedback from the study participants about the system's intuitiveness and immersiveness was positive. Conclusion: We present FIVRS, a fully immersive VR system for skull-base surgery. FIVRS features a realistic software simulation coupled with modern hardware for improved realism. The system is completely open-source and provides feature-rich data in an industry-standard format.Comment: IPCAI/IJCARS 202

Improving Surgical Situational Awareness with Signed Distance Field: A Pilot Study in Virtual Reality

The introduction of image-guided surgical navigation (IGSN) has greatly benefited technically demanding surgical procedures by providing real-time support and guidance to the surgeon during surgery. \hi{To develop effective IGSN, a careful selection of the surgical information and the medium to present this information to the surgeon is needed. However, this is not a trivial task due to the broad array of available options.} To address this problem, we have developed an open-source library that facilitates the development of multimodal navigation systems in a wide range of surgical procedures relying on medical imaging data. To provide guidance, our system calculates the minimum distance between the surgical instrument and the anatomy and then presents this information to the user through different mechanisms. The real-time performance of our approach is achieved by calculating Signed Distance Fields at initialization from segmented anatomical volumes. Using this framework, we developed a multimodal surgical navigation system to help surgeons navigate anatomical variability in a skull base surgery simulation environment. Three different feedback modalities were explored: visual, auditory, and haptic. To evaluate the proposed system, a pilot user study was conducted in which four clinicians performed mastoidectomy procedures with and without guidance. Each condition was assessed using objective performance and subjective workload metrics. This pilot user study showed improvements in procedural safety without additional time or workload. These results demonstrate our pipeline's successful use case in the context of mastoidectomy.Comment: First two authors contributed equally. 6 page

Twin-S: A Digital Twin for Skull-base Surgery

Purpose: Digital twins are virtual interactive models of the real world, exhibiting identical behavior and properties. In surgical applications, computational analysis from digital twins can be used, for example, to enhance situational awareness. Methods: We present a digital twin framework for skull-base surgeries, named Twin-S, which can be integrated within various image-guided interventions seamlessly. Twin-S combines high-precision optical tracking and real-time simulation. We rely on rigorous calibration routines to ensure that the digital twin representation precisely mimics all real-world processes. Twin-S models and tracks the critical components of skull-base surgery, including the surgical tool, patient anatomy, and surgical camera. Significantly, Twin-S updates and reflects real-world drilling of the anatomical model in frame rate. Results: We extensively evaluate the accuracy of Twin-S, which achieves an average 1.39 mm error during the drilling process. We further illustrate how segmentation masks derived from the continuously updated digital twin can augment the surgical microscope view in a mixed reality setting, where bone requiring ablation is highlighted to provide surgeons additional situational awareness. Conclusion: We present Twin-S, a digital twin environment for skull-base surgery. Twin-S tracks and updates the virtual model in real-time given measurements from modern tracking technologies. Future research on complementing optical tracking with higher-precision vision-based approaches may further increase the accuracy of Twin-S

One-stop stroke management platform reduces workflow times in patients receiving mechanical thrombectomy

Background and purposeClinical outcome in patients who received thrombectomy treatment is time-dependent. The purpose of this study was to evaluate the efficacy of the one-stop stroke management (OSSM) platform in reducing in-hospital workflow times in patients receiving thrombectomy compared with the traditional model.MethodsThe data of patients who received thrombectomy treatment through the OSSM platform and traditional protocol transshipment pathway were retrospectively analyzed and compared. The treatment-related time interval and the clinical outcome of the two groups were also assessed and compared. The primary efficacy endpoint was the time from door to groin puncture (DPT).ResultsThere were 196 patients in the OSSM group and 210 patients in the control group, in which they were treated by the traditional approach. The mean DPT was significantly shorter in the OSSM group than in the control group (76 vs. 122 min; P < 0.001). The percentages of good clinical outcomes at the 90-day time point of the two groups were comparable (P = 0.110). A total of 121 patients in the OSSM group and 124 patients in the control group arrived at the hospital within 360 min from symptom onset. The mean DPT and time from symptom onset to recanalization (ORT) were significantly shorter in the OSSM group than in the control group. Finally, a higher rate of good functional outcomes was achieved in the OSSM group than in the control group (53.71 vs. 40.32%; P = 0.036).ConclusionCompared to the traditional transfer model, the OSSM transfer model significantly reduced the in-hospital delay in patients with acute stroke receiving thrombectomy treatment. This novel model significantly improved the clinical outcomes of patients presenting within the first 6 h after symptom onset

TOWARDS VISION-GUIDED SKULL BASE SURGERY

Skull base surgery is a complex procedure due to the critical anatomies only millimeters away from the surgical field. Damaging these critical tissues can lead to life-threatening surgical complications. The limited visibility further exacerbates the situation as most of these tissues are hidden behind the bone. To improve surgical safety and clinical outcome, a vision-based system is built to provide surgeons with more context-situational awareness. The contributions of this dissertation include 1) a set of computer vision algorithms that perform spatio-temporal reasoning of the surgical scene, and 2) simulation technologies that enable advanced visualization of the current surgical progress. The novel computer vision algorithms enable us to perform depth estimation, motion tracking, and surface reconstruction. The simulation technologies include virtual drilling software and a digital twin paradigm. Together, these components build a synergistic ecosystem that has the potential to meet the clinical accuracy requirements as it evolves